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Final  Technical  Report 

Robust  Optimization  of  Large-Scale  Systems:  An  Emerging  New  Technology 
1  October  1991  to  28  February  1995 

John  M.  Mulvey 
Robert  Vanderbei 
Princeton  Uni\er:fity 

This  report  describes  the  research  under  the  support  of  tlie  Air  Force  Grant  (AFOSR-91-0359)  to 
Princeton  University. 

The  primary  research  of  this  project  was  to  develop  practical  techniques  that  give  rise  to  robust 
solutions  to  optimization  problems.  To  achieve  this  goal,  w'e  have  carried  out  research  on  three 
key  topics:  (1)  the  design  of  efficient  algorithms  for  solving  the  resulting  large  scale  optimiza¬ 
tion  problems;  (2)  implementation  strategies  for  several  significant  real-w’orld  applications;  and 
(3)  research  on  tlie  modeling  of  the  stochastic  parameters  via  scenario  generation.  The  results  on 
each  of  these  areas  are  given  below. 


1.  Algorithms  for  Solving  Robust  Optimization  Models 

The  constraints  in  the  basic  robust  optimization  problem  possesses  a  structure  similar  to  the 
constraints  in  stochastic  programming  problems.  They  can  be  depicted  in  rwo  basic  ways:  a 
compact  form,  where  the  non-anticipativity  conditions  have  been  reduced  to  a  minimum  set,  or 
in  a  split  variable  form  where  the  non-anticipativity  constraints  are  modeled  explicitly.  The  lat¬ 
ter  structure  seems  to  be  more  amenable  to  interior  point  algorithms. 

There  are  tw'o  critical  differences  between  robust  optimization  and  the  traditional  stochastic  lin¬ 
ear  programming  model.  First,  robust  optimization  stresses  risk  aversion  via  nonlinear  (convex) 
utility  curves  as  opposed  to  assuming  expected  value  as  the  desired  objective.  Thus,  robust  op¬ 
timization  requires  nonlinear  programming  solvers  for  its  solution.  Second,  robust  optimization 
does  not  require  that  the  constraint  set  for  tlie  second  and  subsequent  stages  be  feasible  for  all 
values  of  the  first  stage  variables.  We  assume  that  the  violation  of  these  constraints  is  handled 
tlirough  a  nonlinear  penalty  function.  These  issues  are  described  in  tlie  report  by  Mulvey, 
Vanderbei  andZenios  [MV2], 

Since  robust  optimization  requires  nonlinear  objectives,  we  focused  on  algorithms  for  solving  tlie 
subsequent  NLP  problem  (with  linear  constraints).  Considerable  progress  has  been  made  in  tliis 
regard.  Wc  have  improved  the  restarting  tecliniques  at  the  beginning  of  each  iteration  of  our 
DQA  algorithm,  thus  reducing  total  execution  time  by  over  40%  [MIS],  More  importantly,  we 
have  been  able  to  achieve  a  higher  degree  of  accuracy. 

In  addition,  we  extended  the  solution  strategy  in  the  interior  point  algorithm  of  Bob  Vanderbei  — 
tlie  LOQO  solver  [VI 1]  -  to  handle  genera)  convex  objectives  (along  with  quadratic  objectives). 
This  result  is  significant  since  the  execution  time  to  solve  the  general  convex  problem  is  only 
slightly  longer  than  for  solving  linear  programs.  We  have  shown  that  robust  optimization  prob¬ 
lems  arc  tlierefore  no  more  difficult  to  solve  than  stochastic  linear  programs.  References  [M2, 
M4,  M6,  M12,  M14,  M20,  M21,  M22,  M23,  V7,  V9]  describes  these  research  results  in  more 
detail. 
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Recently,  we  specialized  ilie  factorization  routine  for  the  structure  inherent  in  robust  optimiza¬ 
tion  models  (the  split  variable  formulation).  See  [NfV"!]  for  details  of  the  computational  tests. 
This  approach  is  considerably  more  efficient  than  the  traditional  multiple  min-degree  ordering 
routine  (over  50  times  faster). 

Our  computational  testing  were  conducted  in  several  locations,  first,  vve  employed  a  network  of 
UNIX  (SGI)  workstations  at  Princeton  University^  Next,  we  ti*aveled  to  several  supercomputer 
companies  in  order  to  test  the  viability  of  LOQO  and  tire  DQA  algorithm  in  parallel  computer 
environments.  This  w'ork  was  conducted  at  Convex  Corp.  in  Dallas,  Texas,  at  Thinking  Ma¬ 
chines  in  Boston,  hiA  and  at  Silicon  Graphics  in  Mountain  View,  CA.  Wc  showed  that  the  very 
large  robust  optimization  problems  can  be  solved  with  both  parallel  and  distributed  computer 
architectures-  Our  larger  test  cases  possess  over  1,000,000  constraints  and  1,700,000  nonlinear 
variables.  These  problems  arc  among  tlie  largest  nonlinear  programs  ever  solved. 


2.  Strategies  for  Implementing  Robust  Opiimization  Models 

We  demonstrated  that  robust  optimization  can  be  used  to  efficiently  model  decision  problems. 
We  contacted  several  groups  developing  planning  models,  including  the  Air  Force’s  Air  Mobil¬ 
ity  Command  (AMC)  at  Scott  AFB,  Bellcore  in  New  Jersey,  US  West  in  Boulder  Colorado,  and 
several  financial  organizations.  For  example,  we  have  worked  with  the  AMC  in  order  to  take 
tlieir  flight  planning  model  (STORM)  and  add  stochastic  scenarios  to  the  problem.  We  devel¬ 
oped  a.  system  for  generating  scenarios  based  on  historical  data  on  tlie  demands  for  cargo  ship¬ 
ments  and  used  tliis  system  to  generate  the  robust  optimization  problems.  This  robust  STORM 
system  has  been  sent  to  a  number  of  stochastic  programming  researchers,  including  Gcrd  In- 
fanger  and  George  Dantzig  at  Stanford,  S.  Sen  and  .lulia  Higle  at  University  of  Arizona,  Roger 
Wets  at  UC  Davis,  and  Alan  King  at  IBM  research  in  Yorktown  Heights,  NY.  We  solved  tlie 
STORM  model  witli  over  200  scenarios  using  our  modest  netivork  of  workstations  at  Princeton 
University.  (See  references  [M2,  M6,  MIA,  Ml 5,  Ml 9].) 


In  addition,  w'e  developed  a  telecommunication  planning  mode!  that  is  robust  witli  respect  to 
network  survivability.  The  goal  is  to  design  the  network  in  a  manner  such  that  tl^e  projected 
calls  can  be  routed,  even  if  part  of  the  network  in  brought  down.  The  test  case  w'as  w'orked  out 
wuth  researchers  at  Bellcore,  Our  design  goal  for  this  problem  is  to  ensure  that  any  one  node 
'could  be  dropped  without  adversely  impacting  tlie  rest  of  the  net\vork.  How’ever,  otlier  design 
goals  could  be  handled  in  the  context  of  additional  scenarios.  This  test  case  has  been  the  focus  of 
both  modeling  and  algoritlimic  research,  (See  references  [M7,  MS].) 


Finally,  developed  some  multi-stage  planning  systems  for  financial  planning,  in  particular  for 
integrating  assets  and  liabilities.  These  problems  are  important  for  large  institutions  attempting 
to  coordinating  their  cashflows  over  extended  periods.  The  use  of  these  integrative  risk  man¬ 
agement  models  should  reduce  risks  to  large  organizations.  For  example,  an  application  w'ouid 
be  the  building  and  costing  of  aircraft  over  time.  Tlie  model  includes  multiple  periods  out  to  40 
years.  These  models  are  discussed  in  references  [Ml,  M3,  M5,  M9,  MIO,  Ml 3,  Ml  6,  Ml 7, 
M18]. 
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3.  Modeling  Stochastic  Parameters  and  Scenario  Generation 

In  these  robust  optimization  models,  an  important  consideration  is  the  handling  of  uncertainty  -- 
for  example,  interest  rales  and  inflation.  The  cost  estimates  will  depend  upon  inflation  and  hence 
the  liabilities  must  be  linked  to  inflation  as  determined  by  the  scenarios.  A  coherent  picture  is 
needed.  We  have  developed  a  scenario  generation  procedure  for  tlte  economic  factors  which  is 
consistent  wdth  respect  to  academic  research  ai-id  is  easily  incorporated  in  a  robust  optimization 
framew'Ork.  Several  meetings  have  been  conducted  wadi  David  Gay  (AT&T)  and  Robert  Fourer 
(Northwestern  University)  in  order  to  extend  the  AMPL  modeling  language  to  automatically 
handle  scenario  generation.  In  addition,  we  have  met  several  people  at  Frontline  systems  to 
show  them  how  to  include  robust  optimization  in  their  spreadsheet  packages.  Frontline  has  im¬ 
plemented  robust  optimization  in  Excel  and  other  major  spreadsheet  vendors  may  do  the  same. 
At  present,  tliese  systems  are  available  as  add-in  routines  to  the  spreadsheets.  A  goal  of  tliis  ef¬ 
fort  was  to  provide  interfaces  that  are  convenient  and  easil}'  managed,  so  tliat  robust  optimization 
can  be  performed  without  a  major  investment  in  time  or  effort.  This  would  improve  the  chances 
of  using  robust  optimization  in  practice. 

Next,  vve  investigated  the  need  for  multi-objective  methods  in  robust  optimization.  It  should  be 
recalled  that  robustness  can  be  achieved  but  there  might  be  additional  costs.  The  cost  depends 
upon  ±e  actual  circumstances  of  tlie  application.  Since  robust  optimization  can  be  used  for  long 
term  planning,  wc  need  to  consider  preferences  over  time.  See  reference  [Mil]  for  a  preliminary 
discussion  of  the  issues  that  are  pertinent  for  individuals  when  planning  their  financial  future. 
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Major  Research  Accomplishments  during  Grant 


1.  Generalized  inferior  point  algorithms  to  handle  convex  objectives  and  deinorisiraied  that  ap¬ 
proach  is  efficient  for  large  scale  problems. 

An  important  consideration  for  robust  optimization  is  modeling  risk  aversion  with  nonlinear 
(convex)  objective  functions.  The  most  imporiant  objective  is  to  maximize  expected  utility  over 
the  range  of  possible  scenarios.  Including  nonlinear  objectives  in  the  interior  point  algorithms 
has  been  quite  important.  We  showed  that  solving  the  robust  optimization  problems  with  convex 
objective  is  only  slightly  more  difficult  to  solve  titan  solving  the  traditional  stochastic  linear  pro¬ 
grams.  Thus,  it  is  likely  that  nonlinear  objectives  will  not  be  a  bander  to  the  use  of  robust  opti¬ 
mization. 


2.  Implemented  the  LOQO  and  DQA  systems  using  several  computer  hard^vare  architectvres, 
including  parallel  and  distributed  computers,  and  increased  the  size  of  solvable  robust  opti¬ 
mization  problems. 

Robust  optimization  problems  grow  quickly  as  a  function  of  the  number  of  scenarios.  However, 
wc  like  to  include  a  wide  range  of  possible  scenarios  to  model  the  stochastic  parameters  in  a  real¬ 
istic  fashion.  Thus,  we  are  faced  the  tradeoff  of  numerical  efficiency  (or  even  solvability)  versus 
modeling  accuracy.  It  is  therefore  important  for  us  to  have  the  ability  to  solve  problems  with  a 
moderate  number  of  scenarios  (at  least  several  hundred  to  several  thousand).  We  demonstrated 
that  tlie  DQA  metliod  is  highly  effective  in  this  environment.  More  recently,  we  showed  that 
direct  solvers  can  also  be  implemented  on  parallel-processing  computers  by  parallelizing  tlie 
factorization  routine.  This  result  might  apply  to  the  more  general  area  of  large  scale  optimiza¬ 
tion,  but  more  research  is  needed  to  determine  the  degree  of  generality^  of  the  results. 


3-  Specialized  interior  point  codes. 

We  developed  a  preordering  routine  that  greatly  reduces  the  time  to  factor  the  (ADA^)  matrix  —  a 
key  step  in  the  interior  point  codes.  The  preordering  is  called  tree  dissection  and  takes  advantage 
of  tlie  staicaire  of  the  split-variable  robust  optimization  model.  See  reference  [MVl]  for  a  dis¬ 
cussion  of  the  preliminary  tests  of  the  concepts. 


4.  Developed  several  real-yvorld  models  of  robust  optimization. 

Robust  optimization  aims  to  be  a  practical  approach  for  solving  planning  problems.  Wc  demon¬ 
strated  the  value  of  robust  optimization  by  developing  and  solving  planning  models  for  real 
problems.  Several  models  have  been  built,  including  a  robust  STORM,  a  telecommunication 
survivability  model,  ajid  several  multi-stage  financial  planning  models.  Included  in  this  work  is 
the  generation  of  die  scenarios  to  represent  the  range  of  likely  uncertainties. 
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